Voice processing is used in many systems to provide voice-command or voice-related services to a user. To do so, typical voice processing systems generate voice profiles of users from a large set of voice samples of that user. The large set of voice samples may be captured using the user's device or otherwise supplied to the voice processing system. The voice samples are often captured and analyzed over long periods of time in order to train the voice processing system.
Voice samples of a user may be locally or remotely processed. For example, in some voice processing systems, voice samples collected from a user are processed on the user's devices and subsequently transmitted to a system outside of the user's devices, for example, a cloud-based system. Alternatively, the raw, collected voice samples may be transmitted to a system outside of the user's device for processing. Either way, user's privacy-sensitive information embedded in, associated with, or otherwise determinable from the voice samples is at risk once the voice samples leave the user's devices. As such, the user's privacy-sensitive information may be compromised and made public via analysis of the voice samples.